A Systolic Multi-Layer and Multi-Line Stereo Vision System for 3D Visual Inspection

This paper describes a new stereo vision system for 3D visual inspection. 2D visual inspection is a typical method to assess a structure visually for corrosion and cracks, as well as other irregularities. However, if we use 3D shape information in addition to the 2D pattern, we can locate defects more clearly and exactly. Stereo vision is a process of recreating 3D spatial information from a pair of 2D images and is one of the most difficult tasks in computer vision due to the ill-posed nature of the problem. We present a new practical stereo vision system, which incorporates a multi-line multi-layer algorithm with a method based on geometrical states in 3D space. In this manner, this system is robust to miscalibration, noise, and variable lighting conditions. In our experiment, the system was attached to an inspection robot and was used to search for weld defects on a metal plate. It showed good results from a pair of input images captured by CCD cameras. Introduction The strain caused by corrosion increases the growth of cracks. Cars, airplanes and several structures are exposed to the danger of cracks and corrosion, and have to be inspected to prevent accidents. Normally, these inspection methods depend on human vision. We present a new real-time stereo matching algorithm that can support 3D spatial information on the surface of the target in addition to the 2D pattern for efficient inspection. It can give 3D information, which is attached to a mobile robot. This increases safety for the inspector as well as creating a more comfortable working environment by enabling the inspector to test visually from a remote place. Using some image understanding algorithms, [1] the areas without cracks, corrosion, or other detectable damage can be coarsely prefiltered. As a result, it can decrease inefficiency due to workers’ fatigue and boredom. A 2D image from a camera can be used to detect surface cracks and surface corrosion using a texture pattern, while the subsurface corrosion area can be detected by surface distortion, called pillowing. Therfore, we need surface shape information to find these types of corrosions. The subsurface can be detected by surface pillowing. Pillowing can be the status which increases in the skin surface height over a region suffering from subsurface corrosion. Our system supporting 3D information can be applied to this area more effectively to observe this characteristic. As human beings can perceive spatial information from two eyes to detect dents, bulges, and depressions, etc., our system can calculate 3D distance information from two images in real time. When the stereo vision system with the varied camera angle is attached to the robot, large stereo matching errors may occur because of the epipolar line estimation problem from mechanical vibration, vergent camera rotation, etc. This has to be solved dynamically. In other stereo vision systems [2, 3, 4], a pair of corresponding pixels in the right and left images can be searched only when a pair of stereo images are rectified. If the exact matching point is not on each scanline, a large error may occur. To decrease the error, a device that controls the position of the camera precisely or a preprocessor of rectification [5] can be used. Because it should dynamically estimate several parameters and models about the epipolar line, lens distortion, etc, this work is cumbersome [6] and can create more difficult Key Engineering Materials Vols. 270-273 (2004) pp 221-226 online at http://www.scientific.net © (2004) Trans Tech Publications, Switzerland Online available since 2004/Aug/15 All rights reserved. No part of contents of this paper may be reproduced or transmitted in any form or by any means without the written permission of the publisher: Trans Tech Publications Ltd, Switzerland, www.ttp.net. (ID: 130.203.133.34-14/04/08,11:29:52) Title of Publication (to be inserted by the publisher) problems. So we present an algorithm apt for VLSI that easily matches in real time without rectification. The total complex system can be implemented on one simple chip. Therefore, it is possible to design a small and simple, low cost system. We use a method that searches the pixel within several lines of one image corresponding to a pixel on the scanline of the other image. By extending the trellis based algorithm [7], we designed a multi-layer system. Because total time complexity is O(N) for each scanline of N pixels, it is possible to process in real time. Multi-layer Trellis Search Algorithm In this section, we present a new trellis based algorithm apt for real time stereo matching without rectification. Fig. 1. The input scanlines for the n layer trellis. Fig. 2. The three layer trellis. We simply assume that this system searches for the pixel within several lines of the left image corresponding to a pixel on one scanline of the right image. It can be changed conversely to the system for one line of the left image and several lines of the right image. The observed image matrix G of N by M size and the m'th scanline m g is defined as follows: